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Abstract — Balancing efficiency of bandwidth use and complex- 
ity of detection involves choosing a suitable load for a multi-access 
channel. In the case of synchronous CDMA, with random codes, 
it is possible to demonstrate the existence of a threshold in the 
load beyond which there is an apparent jump in computational 
complexity. At small load unit clause propagation can determine 
a jointly optimal detection of sources on a noiseless channel, but 
fails at high load. Analysis provides insight into the difference 
between the standard dense random codes and sparse codes, and 
the limitations of optimal detection in the sparse case. 

I. Introduction 

Multiuser detection is the problem of extracting estima- 
tions of multiple sources from a shared communication chan- 
nel 0. In order to most efficiently use the bandwidth a high 
load should be used, and many theoretical results indicate 
achievable capacities are good in this regime for standard 
code classes. However, increasing load forces users to share 
bandwidth and creates a strongly correlated inference problem, 
for which optimal detection may not be possible by practical 
(fast) detectors. Even in the limit of zero noise in the channel, 
multi-access interference may prevent a Jointly or Individually 
Optimal (JO/IO) estimation of the sources. 

Code Division Multiple Access (CDMA) is a method of 
bandwidth allocation in which each of K users is assigned a 
code (si-) by which to modulate a symbol on the bandwidth of 
(3K (M) orthogonal time/frequency blocks (chips), (3 is called 
the load. The scenario of a noiseless multi-access channel is 
examined in this paper for ensembles of sparse random codes. 
The sparse codes examined have the advantage that they can be 
assigned independently at random to all users, and are known 
to have a good performance in Additive White Gaussian Noise 
Channels (AWGNC), combined with Belief Propagation (BP) 
decoding 0, 0, 0. 

Using only Unit Clause Propagation (UCP) all source 
may be determined efficiently for (3 up to some discontin- 
uous transition point beyond which detection by decimation 
becomes suddenly inefficient. 

The noiseless threshold results provide bounds on achiev- 
able detection and may guide the development of algorithms. 
Features of this transition may be relevant to decimation based 
detectors in a variety of sparsely coded noisy channels. For 
AWGNC at standard operating power levels (signal to noise 
ratio ~ 6 — 9dB) the departure of the signal from the noiseless 
case is quite small and threshold behaviour in the noiseless 
system may have a dominating effect on detector performance. 



A. CDMA model and pseudo-random codes 
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Fig. 1 . (colour online) Two different coding schemes are demonstrated with 
K = 5 and load /3 = 1 (right) and 5/3 (left). Connections indicate a non-zero 
transmission by the user on the chip (s M fc =±1). Left: When several users 
are coincident on one chip inference of one variable depends on values of the 
others. Right: Since each user has a unique chip detection is much easier, but 
this scenario requires coordination of user transmissions and small load 0. 

A standard synchronous CDMA model has each user trans- 
mitting a modulated bit 6fe = ±l, the sources interfere to give 
a signal 

y = Y2skbk + u, (1) 

fc 

where uj represents channel noise. Different encodings may 
be represented as graphical inference structures, as shown in 
figure [T] Weakly correlated codes, such as orthogonal codes 
create computationally easy detection problems, at high loads, 
or with poorly chosen codes, detection may be hard. 

For many noise models, including the noiseless limit, a 
maximum capacity can be achieved by minimising overlaps 
in user codes Sk.si (6), orthogonal codes are a solution 
when (3 < 1. However, creating maximum distance codes is 
computationally expensive as j3 increases, and the allocation 
of codes can be inflexible. 

In realistic operating conditions synchronisation of users 
may not be possible, and codes must be made robust against 
a number of phenomena. Furthermore a small loss in capacity 
might be tolerable in order to achieve greater flexibility in the 
code allocation, or efficiency in the detection process. For this 
reason codes sampled independently from code ensemble are 
often considered. Random codes in which every user accesses 
M chips with a unique modulation pattern imply inference 
structures described by dense graphs Q, and these have 
become favoured in theory and practice. More recently it has 
been argued that sparse codes, where each user accesses only 
C (<gC M) chips might have some favourable properties 0, 
particularly due to the efficiency of BP and message passing. 



Analysis by Tanaka [7] suggests that for dense codes in 
the asymptotic case of many user and large bandwidth, limits 
of [3 C = 2.09(1.51) exist above which IO (JO) detection is 
not expected to be efficient due to suboptimal attractors for 
detector dynamics, even in the noiseless limit. In a variety of 
experiments based on BP, and heuristic decoding, the /3 > 1 
regime indeed proves to be difficult |3|, |8| for sparse codes. 
The majority of practical detectors have efficient working 
regimes restricted to (3 < 1. 

In this paper the origins of this hardness at large load are 
investigated for sparse codes in a noiseless channel. UCP is 
found to be sufficient to produce a jointly optimal detection of 
the bits in some range of /?, so that perfect detection is possible 
for loads up to some critical threshold depending on user 
connectivity. The breakdown of the UCP detector is sudden 
and leaves a residual problem without an obvious solution, 
the residual problem has many properties characteristic of hard 
constraint satisfaction problems 0. 

B. Sparse random codes 

The set of codes examined are sparse so that each user 
transmits on only a fraction C/K of chips, Binary Phase Shift 
Keying (BPSK) is used so the marginal is 

P( v A C\ C (5(s 0-1) + 8(s „ fc + l)) 
P{s flk ) = \l - — J 5( Sfik ) + — ^ ^ , 

(2) 

each non-zero transmission is a binary modulation ±1. The 
degeneracy problem central to this paper can be avoided by 
choosing another modulation pattern, but degeneracy becomes 
a problem for such schemes as soon as realistic noise is 
introduced. 

In the Poissonian ensemble each chip is accessed in an inde- 
pendent manner, unfortunately some users end up transmitting 
on no chips in this ensemble. In the more practical Regular 
ensemble all users access exactly C chips so that 

M M 

?(4)«^( V ) 2 -C)[]PM. (3) 

f-i—1 /I— 1 

In the limit of large M the case C — 0(1) has properties 
distinguishable from the dense case due to dilution effects. 

II. Detection by Unit Clause Propagation 

The inference problem in the noiseless channel consists 
of examining the signal and determining an estimation of 
the sent bits (b) consistent with the model Q. The value 
of some bits might only be determined by looking at the 
entire signal, but other bits may be implied from only one 
chip. Each chip determined by L user contributions, with 
signal y, can be interpreted as a logical clause. Some clauses 
allow degenerate solutions when considered in isolation, others 
imply unique solutions. If a user k is the only transmission on 
chip /i for example, then y fl = ±l and any consistent estimate 
requires b \ — s^U^ regardless of other chips on which 
k transmits. The logical implication is a unit/atomic clause 
in the variable k. Similarly a chip combining L interfering 



transmissions determines all the incident users (through L 
unit clauses) if the signal is extremal y = ±L. Otherwise the 
solution is degenerate, allowing several consistent assignments 
of incidents bits. 

UCP produces an estimate by decimating variables con- 
tained in unit clauses and is a central process in many 
complete detectors. By iteratively removing the unit clauses 
the remaining degenerate clauses, which form a simplified 
inference problem (residual graph), may be modified and 
create additional unit clauses. If insufficient unit clauses exist 
at some point in the algorithm, one can guess a value at 
random, or using some more advanced inference, and UCP 
checks the logical implications of this guess. A consequence 
of guessing is that one may produce contradictory unit clauses 
at some later point in the algorithm, a consistent (JO) solution 
is not possible where contradictions occur. However, guessing 
may not need to coincide exactly with the bit sequence for a 
JO detection, it may be that several solutions exist for a given 
signal. This is an undesirable scenario for practical purposes, 
but possible at high load. 

The initial set of unit clauses is found by taking all chips 
for which Ufi — + J2k \ s ^k\ an d converting each to |y„| unit 
clauses. The set of unit clauses f2 + is populated, while this 
set is non-empty, and includes no contradictions, the algorithm 
proceeds deterministically in two steps. 

In the first step a variable is decimated: Select a variable 
k represented in f2 + , set its value b \ according to the unit 
clause(s) and remove unit clauses in variable k from i~2 + . 
Reevaluate the signal on all chips on which user k transmitted: 
t/ m — ^ j/ M — s^kbk, and then remove (set to zero) s^k for all 
\i. Variable k is removed from the problem leaving a residual 
graph. Let X be the the number of variables assigned in this 
way, the decimation time, so that X — > X + 1 in this step. 

In the second step modifications to the residual graph are 
considered, chips formerly degenerate may now be informa- 
tive. For every chip modified in the first step check whether 
J2k \ s tJ.k\ = +y^, if this condition is met then the inference is 
no longer ambiguous and a unit clause is created and added 
to the set i~2 + , for each non-zero s^k- 

If f2 + is empty then a variable is chosen and a unit clause 
is created and added to the set f2 + . In the simplest scenario 
the variable {k) and assignment 6fc = ±l are chosen uniformly 
at random. Once the first such guess is made it is necessary 
to keep track of contradictions. If a unit clause is added to 
the set f2 + that is in contradiction with an existing unit clause 
the algorithm may proceed to termination by ignoring the new 
unit clause, but the estimate obtained is not JO. 

The algorithm completes after K decimations, either with 
the unique JO estimation (if no guesses were required), a JO 
estimation (if guesses were required, but no contradictions 
encountered), or an approximation (if contradictions were 
encountered). If a JO solution is required the algorithm may be 
modified so as to back-track and reevaluate previous guesses 
when contradictions are encountered. 

An example of the algorithm in action for the third scenario 
is shown in figure [2] for an experiment with C = 3, /3 = 2, 
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Fig. 2. (colour online) A typical decoding by the proposed method of 
a regular code transmission (C = 3, /3 = 2, K = 1000) is shown. Up 
to decimation time X£,K the process is deterministic, > 0, and 

errors are absent. Beyond this time a fraction of variables are determined 
by guesses, the bit error rate begins to increase linearly once the first guess 
is made. At decimation time xqK contradictions appear in indicating 
incorrect guesses. Both critical times exist in this example, it is also possible 
that the algorithm runs without requiring any guesses, or without producing 
contradictions, as occurs for smaller load. 



a random bit sequence, and codes sampled from the regular 
ensemble Q. Up to a time xdK the algorithm is deterministic, 
and up to a time xc<K no contradictions are encountered. 
Between xdK and xcK a small fraction of variables must 
be guessed, the Bit Error Rate (BER) increases linearly in 
expectation from the first guess. Many contradictions arise 
later in the algorithm, implied by the small number of guesses. 

III. Asymptotic Results 
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Fig. 3. (colour online) The initial inference problem consists of variables 
distributed at random within L-y clauses. In order to describe the mean prop- 
erties of the algorithm at the initial conditions, and through the deterministic 
phase up to xjjK decimations, <E>;(X) and Q C (X) are sufficient statistics. 
These are respectively the number of clauses of length I (ignoring clauses 
larger than L max , a sufficiently large upper bound) describing the residual 
graph, and the number of variables appearing c times in the set of unit clauses 
(0+), a sufficient description of f2-|_. 



Experimentation indicates a self-averaging effect for large 
K that can be analysed throughout the deterministic stage of 
the algorithm (X/K < X£>). This is possible by assuming 
a concentration process in the various extensive statistics of 



the residual graph |5|, [10], and the extensive population of 
distinct unit clauses, the important extensive properties are 
illustrated in figure [3] The Poissonian ensemble is analysed 
for brevity, the Regular ense mble derivation is more subtle 
and outlined in Appendix 



In the Poissonian ensemble the number of chips with I 
incident users is Poissonian distributed, parameterised by L 
(= Cj3), a finite upper bound is assumed for the largest relevant 
chip L max . Of chips containing I contributions, a fraction 
are non-degenerate for / > 1. The remaining clauses 
are degenerate, the number of degenerate clauses of length 
I is called $i(X). The fraction of signals taking value y for 
given I in the residual graph is Binomial in expectation at time 
X = 0, and at later decimation times is found to be dependent 
only on I given $i(X) (a proof by iteration is possible). The 
initial condition is 

exp(-L)L r 
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For the Poisson dynamics the only other statistic required is the 
number of un-decimated variables that are not represented in 
the set of unit clauses, ilo(X). For the Poissonian ensemble 
this number, at X = Q, is the probability of a zero in the 
Poisson distribution parameterised by the total number of 
variables incident on chips of type y = ±L. 
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The number of times a variable is represented in the set 
of unit clauses, and the number of times it appears in the 
graph are conditionally independent at time X — given the 
extensive statistics. Therefore a variable decimated from the 
set of unit clauses is incident in chips of length I a number 
of times proportional to length I, and inversely proportional 
to the number of remaining variables K — X. This clause 
may become either a smaller clause, or a set of L — 1 unit 
clauses, the later with probability z/ (independent of X). The 
population $>i(X) is modified according to 
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(6) 
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except for / = L max where the second term is absent. This 
applies intuitively at the first decimation time X = 0, but it is 
also true at all X. Variables which become unit clauses through 
the process of decimation do so at a rate proportional to their 
connectivity, and are represented in the residual graph with a 
frequency determined by their excess connectivity distribution. 
For the Poisson ensemble these are both the same and whether 
a variable is represented zero or several times in the set of unit 
clauses does not effect its distribution in the residual graph. 

As the clauses are reduced a fraction become new unit 
clauses, determined by z\ and <&;. At time X, there are K — X 
variables remaining, of which ilo(X) are unrepresented in a 

'This appendix is included here but absent in the Physcomnet submission 
for reasons of page limitation. 



unit clause. Since the new unit clauses represent a random 
sample of the variables, £Iq(X) has an expected change 
proportional to the fraction of variables it contains and the 
total number of new unit clauses 



Oa(X + l) = n (X)- 
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Writing the various terms at O(K), (X = xK,£l — 
(jJqK,$i = 4>iK), and Taylor expanding the left hand sides 
up to first order in h leads to differential equations 

d<M^) 1 . , , , + 



ax 1 — x 1 — x 



z i+ i)(j)i +1 (x) , (8) 



for 2 < I < L max , 
and 



for I = L r . 



the second term is absent, 
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These equations can be solved to determine the extent of 
the critical domain. It is necessary to find the first point in the 
interval [0, 1] where uiq{xd) = 1 — %d, this is the point where 
the set of unit clauses becomes empty. For the Poissonian code 
the equation dSl is solved by a set of polynomials of the form 
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(10) 



where the coefficients can be determined iteratively from the 
equation for I — L max , incorporated additional boundary 
conditions for smaller Using these solutions (|9| may be 
solved as an exponential form. Roots of ljq(xd)="1 — xo in 
the interval [0, 1) can be determined numerically. 

Unfortunately the equation for the regular ensemble are 
not so easily solved. One must keep track of the dynamics 
for variables of different degeneracies in S~2 + , the number of 
variables appearing c times as unit clauses is Q, C (X), where 
c runs from to C. A variable present in f2 + occurs less 
frequently in the residual graph, which must be accounted for 
in and the new unit clauses contribute to the populations 
{il c (X)} in a non-linear fashion. Including initial conditions 
one can again write a set of differential equations, but these 
must be solved by numerical integration. Fortunately the 
dynamics are smooth and fourth order Runge-Kutta methods 
seem to produce a good determination of the critical point, 

ZD- 
Figure [4] demonstrates the results of the numerical inte- 
gration for the regular case with C — 3 at various loads by 
comparison with experiments on systems with X=1000 and 
K = 10000. Similar excellent agreement was found for the 
Poissonian ensemble |8|. 

IV. Computational Hardness 

Figure |4] shows that there is a discontinuous jump in xd 
as (3 increases, the guessing stage goes from being empty to 
forming a large fraction of the algorithm run-time, and the 
Terminal Residual Graph (TRG) goes from empty at xd to 
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Fig. 4. (colour online) The asymptotic and experimental results for 
xo indicates a first order transition in /3 when C = 3. The mean excess 
connectivity of variables remaining in the graph when xjj < 1 is large 
(inset), implying the terminal residual graph is not tree like. The experimental 
curves indicate the median, first and third Quartiles for these values based on 
1000 (100) samples for for IfclOOO (10000), typical cases are observed to 
concentrate on the asymptotic prediction. 



a giant loopy graph. At small (3 and C greater than 2 there 
is only a deterministic phase and determining the unique JO 
detection (also IO) is trivial (xd =Xc = 1). Similarly below 
the percolation threshold, the solution ceases to be unique, 
finding one of the degenerate solutions remains easy (x£><1, 
xq = 1). Finally at large (3 and C, the algorithm involves a 
non-deterministic phase, and generates contradictions {x£> < 
xc < 1). This latter regime dominates for large C for finite 
(3. At large (3 and finite C one can expect degeneracy in the 
solutions, but close to the transition the embedded solution is 
expected to remain the unique JO solution, only difficult to 
find by UCP The three regimes can be bounded in a phase 
diagram as shown in figure [5] Up to the range of (3 shown it is 
found that perfect decoding (BER=0) coincides with xc = l, 
except at small C. The JO solution is unique and trivial to find 
at small load. The case of a Poissonian ensemble is examined 
briefly in Appendix [A] 

Figure [2] shows properties typical of the regime in which 
the non-deterministic method exists. For computational com- 
plexity reasons the gap ip-io is most important. If this 
is not asymptotically 0, then with high probability O(K) 
guesses have been made in reaching a contradiction. The cost 
of backtracking, reevaluating 0(2 ) combinations to find the 
correct branch, is not feasible and UCP becomes an efficient 
JO estimator. 

In order to understand the algorithm failure it is useful to 
consider the TRG structure. In the regular case the graph 
consists of a C-core, every variable is connected to exactly 
C degenerate chips, each chip implying a constraint. These 
constraints are of many types, comparable to 2-XOR, 1-in- 
3 SAT iflOl , and others. The clauses are locked - at least 
two variables in any clause must be changed to go from one 
solution to another. Random Constraint Satisfaction Problems 



(CSP) of a similar specification have been much studied in 
physics (9), iflOl - Increasing the ratio of logical constraints 
to variables (1/jJ) often leads to a sharp transition in the 
ability to find any solution (SAT transition) buffered by a 
computationally hard regime. 

The case studied is topologically similar to a CSP, but the 
embedded solution changes the nature of the transition ifTTl . 
and it is difficult to establish the extent of this effect. There is 
always a (BER= 0) JO solution, but one may asymptotically 
expect a sharp transition from a computationally easy regime 
for JO detection, to a computationally difficult one in which 
the solution is still unique, and then towards regimes with 
many solutions (poor performance even with an ideal detector). 
It may be argued that the transition is a peculiarity of the UCP 
decoder and the BPSK modulation scheme (which introduces 
degeneracy). It also seems likely that for TRGs with unique 
solutions methods such as BP decimation may be successful 
beyond the UCP threshold in typical case with some high 
probability. Certainly the critical j3 decreases with C, whereas 
algorithms are known to perform well up to < 1 in practice. 
However, for small C the thresholds for j3 outlined in this 
paper seem to match quite well transitions in the equilibrium 
solution spaces for noisy systems |4|, which may indicate a 
common phenomena. 

V. The role of channel noise 

The effect of noise might be considered for an erasure 
channel without fundamentally changing the analysis fl2l . In 
an erasure channel a fraction of chips are removed, guessing 
becomes necessary and degeneracy is introduced. For the 
Poissonian ensemble the random erasure of chips is equivalent 
to working with a new code parameterised by larger (3. 
However, for the sparse code any small number of erasures 
has an important immediate effect: the TRG 2-core cease 
to be the entire graph and there is no longer a unique 
solution (BER=0). The Poissonian code development scheme 
of figure [6] is applicable. Nevertheless, a sharp UCP transition 
in j3 is still observed between a computationally easy tree-like 
TRG (analogous to the ijj = 1 scenario) and the non-trivial 
loopy TRG, except for small C and/or many erasures. 

Discontinuous transitions in optimal, or heuristically deter- 
mined, bit error rates are common with increasing noise in 
CDMA detection models. Erasures within the noiseless frame- 
work introduce additional degeneracy in the solution space. It 
seems reasonable to expect that a critical number of erasures 
might cause a discontinuous jump in the BER applying to a 
typical jointly optimal solution, related to the solution space 
on the core of the TRG. An equilibrium statistical mechanics 
approach may be most effective at uncovering such features. 

VI. Conclusion 

Unit clauses propagation is an important component in 
many solution finders and seems particularly well adapted for 
sparse CDMA, providing a guaranteed jointly optimal perfor- 
mance across a range of parameters. This high performance 
may be established in typical case by analytic methods. In 
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Fig. 5. (colour online) Curves represent thresholds in the median of 1000 
samples for systems of K = 10000. The regular ensemble is presented, 
interpolating linearly (users of connectivity C and C + 1) where C is not 
integer. The regular ensemble produces unique solutions with C greater than 
2 and (3 small, and when C < 2 (below the percolation threshold) completes 
without contradictions to produce one of a degenerate set of solutions without 
contradictions. The contours in Bit Error Rate indicate a sharp transition 
in estimate quality as f3 increases. The limit of the regime with BER= 
is coincident with the line indicating presence of contradictions above the 
percolation threshold. 



Fig. 6. (colour online) The initial problem has unit clauses and a residual 
graph. Information is transferred from the former to the latter by UCR Two 
different inference paths correspond to the Poissonian (a) and Regular (b) 
problems. With erasures in the regular channel (a) becomes relevant, (a) In 
the Poissonian ensemble there is always an ambiguous stage due to presence of 
disconnected variables, but is computationally easy if only trees are present in 
the residual graph at x-q (TRG). There are no algorithmic methods guaranteed 
to efficiently find solutions in worst cases where a giant component/core is 
present, (b) In the regular ensemble the algotihm either terminates with only 
a deterministic stage, or halts at a 2-core (graph without leaves). Solutions 
must be isolated in (b) due to the locked nature of clauses, in graphs with 
leaves (a) some additional freedom may apply. 
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this paper it is argued that the failure of UCP in the noiseless 
case for large systems, due to multi-access interference at high 
load, characterises an asymptotically limiting regime for all 
fast algorithms due to fundamental topological and solution 
space features of the inference problem. 

Understanding the residual graph problem in greater detail 
might provide insight for sparse codes and, if solved, a basis 
for selection of principled detection methods at high load. A 
basis for this analysis may be found in recent statistical physics 
research on locked constraint satisfaction problems. 

Acknowledgment 

This research is partly supported by the Research Grants 
Council of Hong Kong (Grant No. HKUST 604008). 

References 

[1] S. Verdu, Multiuser Detection. New York, NY, USA: Cambridge 
University Press, 1998. 

[2] M. Yoshida and T. Tanaka, "Analysis of sparsely-spread CDMA via 
statistical mechanics," in Proceedings - IEEE International Symposium 
on Information Theory, 2006. (Seattle). Piscataway, NJ, USA: IEEE, 
2006, pp. 2378-2382. 

[3] A. Montanari, B. Prabhakar, and D. Tse, "Belief propagation based 
multiuser detection," in 43rd Annual Allerton Conference on Commu- 
nication, Control and Computing 2005. (Monticello). Red Hook, NY, 
USA: Curran Associates, Inc., 2006. 

[4] J. Raymond and D. Saad, "Sparsely spread CDMA - a statistical 
mechanics-based analysis," /. Phys. A, vol. 40, no. 41, pp. 12 315- 
13 334,2007. 

[5] C. Deroulers and R. Monasson, "Criticality and universality in the unit- 
propagation search rule," Eur. Phys. J. B, vol. 49, no. 3, pp. 339-369, 
2006. 

[6] M. Rupf and J. Massey, "Optimum sequence multisets for synchronous 

code-division multiple-access channels," IEEE Trans, on Info. Theory, 

vol. 40, no. 4, pp. 1261-1266, 1994. 
[7] T. Tanaka, "A statistical-mechanics approach to large-system analysis 

of CDMA multiuser detectors," IEEE Trans, on Info. Theory, vol. 48, 

no. 11, pp. 2888-2910, Nov 2002. 
[8] J. Raymond, "Typical case behaviour of spin systems in random graph 

and composite ensembles," Ph.D. dissertation, Aston University, Birm- 
ingham, UK, November 2008. 
[9] L. Zdeborova and M. Mezard, "Locked constraint satisfaction 

problems, " Phys. Rev. Lett, vol. 101, no. 7, p . 078702, 2008. [Online]. 

Available: http://link.aps.org/abstract/PRL/vl01/e078702 
[10] J. Raymond, A. Sportiello, and L. Zdeborova, "The phase diagram of 

random l-in-3 satisfiability problem," Phys. Rev. E, vol. 76, no. 1, p. 

011101, 2007. 

[11] L. Zdeborova and F. Krzakala, "Quiet planting in the locked constraint 

satisfaction problems," 2009, arXiv:0902.4185. 
[12] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman, 

"Efficient erasure correcting codes," IEEE Trans, on Info. Theory, 

vol. 47, no. 2, pp. 569-584, 2001. 

Appendix 

The mean dynamics for the regular user connectivity en- 
semble are developed along similar lines to those presented in 
the main text, expanding the explanation on certain points. At 
decimation time X typical ensemble dynamics are described 
sufficiently by $i(X) and Cl c (X). The latter term describes the 
number of variables present c times in the set of unit clauses 
(including those that are absent c — 0). During the deterministic 
phase unit clauses exist, the critical criteria demarking this 
algorithmic stage is 



Selecting a variable independently of its degeneracy, leads to 
a decrease in the number of degenerate unit clauses, therefore 
decimation contributes to the dynamics according to 

n c (x) 



A«O c (X) 



(12) 



for all non-zero c. The number of residual graph variables 
associated to a unit clause depends on its degeneracy, in the 
regular ensemble a variable appears C times in the unit clauses 
and residual graph, so the average probability an instance 
of a variable in the residual graph corresponds to a variable 
decimated from the set of unit clauses, relative to a uniform 
sampling of the unit clauses, is 

c c 
e(X) = J^iC ~ c)n c (X)/ J^iC - c)n c (X) . (13) 



Variables decimated appear less frequently in the residual 
graph than the mean. 

A clause population may grow with the removal of a vari- 
able, as larger degenerate clauses become smaller degenerate 
clauses, or become smaller as the population is reduced. A 
variable is coincident with a clause in proportion to its size 
the dynamic in expectation is 



A* z p0 = 



K-X 



(-!$,(I) + (l- 2(+1 )(I + l)$ W (I)) . 

(14) 

The dependent term is absent for the special case 

I = L max . In the large system limit L max is not finite, but 
choosing a finite value generates only a very small systematic 
error. The factor zi = 2 2 ~'/(l — 2 / ~ 1 ), since it is the probability 
the reduced chip contains only one true or one false literal 
(modulated variable), y = L — 2 or 2 — L, multiplied by the 
probability that the same literal is selected. Thus it is the 
probability a degenerate clause becomes non-degenerate when 
a typical variable is decimated from a clause of length I. The 
distribution of clauses of type L, y is Binomial at time X = 0, 
and can be shown by iteration to remain so at later times for 
the degenerate clauses, hence the simple form of zi. 

Each reduction of a large clause to a non-degenerate clause 
creates I — 1 unit clauses. These new unit clauses coincide 
with variables in the set <I> C in proportion to C — c, for 
example a variable C times degenerate in the unit clauses 
cannot be represented in the graph (since it only appears C 
times in the problem before decimation). The probability a 
new unit clause is coincident with a variable of degeneracy 
c relative to the fraction of variables with degeneracy c is 
f c {X). Each coincidence results in variables of degeneracy c 
being transformed into variables of degeneracy c+1, therefore 



A( 2) n c (x) = [-/ c (x) 3 ^- + / c _ 1 (x)^ f 

defining /_i(X)=0 and otherwise 

(C-c)Q c (X) Eto n c(X) 



x 



n (x) < k- x 



(ii) 



MX) 



£f=o(c- 



c)n c (x) n c (x) 



(15) 



(16) 



The equations (12i and ( p~5] > combine to produce a small 
change in the population il(X), infinitesimal by comparison 
with the populations up to time xd- Similarly the populations 
of clauses of length I are in expectation order K, but the 
changes due toflilare 0(1). Taking X = xK, $i(X) = K<j)i(x) 



and VL c (X)=Ku) c {x), writing A$(X) as a Taylor expansion 
and keeping only leading orders in K leads to a differential 
description of dynamics 



d(f>i(x) e(x) 



dx 1 — x 
Similarly an equation may be written 



{Ifaix) + (1 - z(l + + l)<f> l+l (x)) . 

(17) 



dujp(x) _ f (x)u> (x) 
dx (1 — x) 



1-x 



f>[f(l-l)]^(x) 



1=2 



(18) 

and for each of the variables in the unit clauses, according to 
their degeneracy 



=(*) 



da; 



e(a) 



= (g) 



f c - 1 {x)^ c - 1 (x)-f c (x)u c (x) 



i e (x) 



(l-x) 



(19) 

As in the previous Poissonian ensemble a set of differential 
equations is evident, the distinction is in the necessity of 
the factors (fT6l)(fT3|l, which can be taken as 1 to recover the 
Poissonian case, and in the need to include information on 
the degeneracy of variables to calculate typical properties. 
The initial condition for are identical to the Poissonian 
case due to the extra factor e(x) these equations do not 
have a concise solution. Initial conditions on f2 c |5]l differ 
from the Poissonian case, in the new model it is a Binomial 
distribution parameterised by 0, and the fraction of all possible 
CK unit clauses immediately revealed by degenerate clauses 
(the exponent of <[5j divided by C). 

The treatment for the regular case, assuming some up- 
per bound in the variable connectivity (0) and the chip 
connectivity (L) provides a basis for generalising to more 
complicated ensembles, although the regular and Poissonian 
ensembles seem most natural in the context of CDMA. The 
possibility to optimise ensembles based on the ability of UCP 
to decode is one application, similar to the application of UCP 
in optimisation of Linear and Fountain Codes 1121 . Solution of 
the equations requires numerical integration, but results with 
simple Runge-Kutta methods were found to be sufficient and 
in remarkably agreement with experiment (figure [4]), verifying 
assumptions of the method. 

A figure analogous to figure [7] is shown for the Poissonian 
ensemble. Unlike the regular ensemble, the Poissonian ensem- 
ble nowhere results in a JO solution of BER= 0. This is due 
to the inhomogeneity in user connectivity, of course only half 
of the unconnected users (of which an extensive number are 
present) may be inferred correctly for example. There is a 
transition between cases for which one of many JO solutions 
is found (without contradictions), and one in which no JO 
solution is found by the proposed method. A sharp transition 
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Fig. 7. (colour online) A figure with statistics and parameterisations 
equivalent to figure [3] but applied to a Poissonian ensemble, is presented. xr> 
is everywhere less than one, there exists three regimes distinguished by the 
BER of solutions and xc, similar qualitatively to those of the regular code. 
Algorithms terminating without contradictions have low BER, except near 
and below the percolation transition. When xq departs from 1 a substantial 
increase in BER of solutions is found. 



is again seen in the structure of the TRG from a tree to a large 
graph containing loops at the threshold time X£>. 

The Poissonian ensemble is not a good candidate for coded 
transmission. However, a regular code in combination with an 
erasure channel might have many properties increasingly sim- 
ilar to a Poissonian coded system as the noise level increases. 
This is observed in experiment and asymptotic analysis. 

A different ensemble with regular chip-connectivity and 
Poissonian user-connectivity was considered in |8|. This study 
demonstrated that when the chip connectivity is exactly 3 an 
optimal solution is always computationally easy to determine 
at any load. However, the transmission properties are not 
favourable for such a regime. This special case demonstrates 
the a limit with dominant dilution effect. 



